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DETAILED ACTION 
Abstract 

1 . Applicant is reminded of the proper language and format for an abstract of the disclosure. 
The abstract should be in narrative form and generally limited to a single paragraph on 

a separate sheet within the range of 50 to 150 words. It is important that the abstract not exceed 
150 words in length since the space provided for the abstract on the computer tape used by the 
printer is limited. The form and legal phraseology often used in patent claims, such as "means" 
and "said," should be avoided. The abstract should describe the disclosure sufficiently to assist 
readers in deciding whether there is a need for consulting the full patent text for details. 

The language should be clear and concise and should not repeat information given in the 
title. It should avoid using phrases which can be implied, such as, "The disclosure concerns," 
"The disclosure defined by this invention," "The disclosure describes," etc. 

As noted above, a single paragraph for the Abstract is stated. Appropriate correction is 
required. 

Claim Objections 

2. Claim 2 is objected to because of the following informalities: -one lower level 
HighlightLevel DSs — should be corrected as -one lower level HighlightLevel DS — (without an 
"s" for singular, not plural). Appropriate correction is required. 

3. Claim 4 is objected to because of the following informalities: claim 4 should be at the 
start of a paragraph itself, not a continuation of claim 3 as presently cited. Correction is required. 
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Claim Rejections - 35 USC 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by 
another filed in the United States before the invention thereof by the applicant for patent, 
or on an international application by another who has fulfilled the requirements of 
paragraphs (1), (2), and (4) of section 371(c) of this title before the invention thereof by 
the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act of 1999 
(AIPA) and the Intellectual Property and High Technology Technical Amendments Act of 
2002 do not apply when the reference is a U.S. patent resulting directly or indirectly from 
an international application filed before November 29, 2000. Therefore, the prior art 
date of the reference is determined under 35 U.S. C. 102(e) prior to the amendment by the 
AIPA (pre-AIPA 35 U.S.C. 102(e)). 

5. Claims 1 -32 are rejected under 35 U.S.C. 102(e) as being anticipated by Sezan et al. 
(U.S. Patent No. 6,236,395 Bl/ or "Sezan" hereinafter). 

Regarding claim 1, Sezan discloses "a Hierarchical Summary Description Scheme (DS) 
for describing a video summary, the HierarchicalSummary DS comprises at least one 
HighlightLevel DS which is describing highlight level, wherein said HighlightLevel DS 
comprises at least one HighlightSegment DS which is describing highlight segment information 
constituting the summary video of the highlight level", i.e., a highlight summarizer 78 describes 
highlight level and highlight segment information, which helps to form a summary video of the 
highlight level within a hierarchical summary description scheme (Fig. 1/item 18 for a program 
summary description scheme. Fig. 3/item 78 for a highlight summarizer. Fig. 13 for a 
hierarchical structure of program description scheme with the use of segment description scheme 
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and event description scheme; see col. 1/lines 55-67 for program summary description scheme 
DS, col. 4/line 40 to col. 5/line 32 for the highlight levels, col. 27/lines 13-43 for segment 
description scheme &. col. 27/line 55 to col. 28/line 9 for events addressed). 

As for claim 2, in fiirther view of claim 1 , Sezan discloses "wherein said HighlightLevel 
DS is composed of at least one lower level HighlightLevel DS", i.e., low level features from 
program description scheme are used for automatically analysis methods (col. 8/lines 18-29) 

As for claim 3, in further view of claim 1, Sezan ftirther discloses "wherein said 
HighlightSegment DS comprises a VideoSegmentLocator DS which is describing time 
information or video itself of said corresponding highlight segment", i.e., a key frame view 
serves as video segment locator for providing time stamps or other video information is included 
within the highlight segment (col. 4/hnes 55-67). 

As for claim 4, in fiirther view of claim 3, Sezan further discloses "wherein said 
HighlightSegment DS fiirther comprises hnageLocator DS which is describing the representative 
frame of said corresponding highlight segment" (Figs. 7-10 shows examples for describing the 
concerned issue for frame view, highlight view etc.). 

As for claim 5, in fiirther view of claim 3, Sezan fiirther discloses "wherein said 
HighlightSegment DS fiirther comprises SoundLocator DS which is describing the representative 
sound information of said corresponding highlight segment", i.e., audio and video or namely 
audiovisual inforination is included within the highlight segment (col. 5/lines 10-32 for audio 
annotation, and col. 9/line 34 to col. 10/line 37 for an example for using the highlights with audio 
and visual information). 
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As for claim 6, in further view of claim 3, Sezan discloses "wherein said Highlight 
Segment DS further comprises ImageLocator DS which is describing the representative frame of 
said corresponding highlight segment and SoundLocator DS which is describing the 
representative sound information of said corresponding highlight segment" (see claims 4 and 5 
above). 

As for claims 7 and 8, in further view of claim 4, Sezan further discloses "wherein said 
ImageLocator DS describes time information or image data of the representative frame of video 
interval corresponding to said corresponding highlight segment" and "wherein said 
HighlightSegment DS fiirther comprises Audio SegmentLocator DS which is describing the 
audio segment information constituting an audio summary of said corresponding highlight 
segment" (Fig. 16 for image and audio segment information with corresponding time 
information and audio summary as well as frame of video interval addressed, col. 27/lines 13-43; 
and col. 5/lines 10-32 for audio annotation). 

As for claim 9, in fiirther view of claim 8, Sezan further discloses "wherein said 
AudioSegmentLocator DS describes time information or audio data of the audio interval of said 
corresponding highlight segment" (Fig. 16 shows time reference according to key frame scheme 
and segment description scheme, with the understanding that audiovisual information is included 
as discussed in claim 5 above). 

As for claim 10, in further view of claim 1, Sezan fiirther suggests "wherein said 
HierarchicalSummary DS includes SummaryComponentList describing and enumerating all of 
the SummaryComponentTypes which is included in the HierarchicalSummary DS", i.e., 
identification of key frames, segment definitions, representative of the video, individual shots or 
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scenes, thumbnail set of frames, highlight definitions etc. are used for identifying the list of 
components in the program description summary (col. 4/line 40 to col. 5/line 9). 

As for claim 1 1, in further view of claim 10, Sezan further suggests "wherein said 
SummaryComponentType includes keyframes representing the key fi-ame summary composed 
of representative frames, keyVideoClips representing the key video clip summary composed of 
key video segment' sets, keyEvents representing the summary of the video interval 
corresponding to either the event or the subject, keyAudioClips representing the key audio clip 
summary composed of representative audio intervals' sets, and unconstraint representing the type 
of summary defined by users except for said summaries" (as discussed in claim 1 1 above, see 
col. 4/line 40 to col. 5/line 3 1 for further details on video clips and audio clips — or audio 
annotations). 

As for claims 12-13, in further view of claim 1, Sezan further suggests "wherein said 
HierarchicalSummary DS includes SummaryThemeList DS which is enumerating the event or 
subject comprised in the summary and describing the ID and then describes event based 
summary and permits the users to browse the summary video by the event or subject described in 
said SummaryThemeList" and "wherein said SummaryThemeList DS includes arbitrary number 
of SummaryThemes as elements and said SummaryTheme includes an attribute of id 
representing the corresponding event or subject" (Figs. 13-15 and Fig. 16, and col. 26/line 29 to 
col. 27/line 43 for event or subject and theme together with IDs for corresponding event or 
subject). 
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As for claim 14-16, in further view of claim 13, Sezan discloses "wherein said 
SummaryTheme further includes an attribute of parentID which is to describe the id of the event 
or subject of the upper level"; "wherein said HighhghtLevel DS includes an attribute of theme 
IDs describing said attribute of ids of common events or subjects if all of the HighlightSegments 
and HighlightLevels which are constituting corresponding highlight level have common events 
or subjects"; and "wherein said HighlightSegment DS includes an attribute of theme IDs 
describing said attribute of ID and describes the event or subject of the corresponding highlight 
segment" (Figs. 13-16 & Fig. 19, and col. 27/line 13 to col. 28/hne 9 with highlight segment and 
theme IDs addressed). 

Regarding claim 17, Sezan discloses "a computer-readable recording medium where a 
HierarchicalSummary DS is stored therein, the HierarchicalSummary DS comprises at least one 
HighhghtLevel DS which is describing highlight level, wherein said HighhghtLevel DS 
comprises at least one HighlightSegment DS which is describing highlight segment information 
constituting the summary video of that the highlight level, wherein said HighlightSegment DS 
comprises VideoSegmentLocator DS describing time information or video itself of said 
corresponding highlight segment" (see claims 1-3 above, with a computer readable recording 
medium as shown in Fig. 2 and col. 7/line 50 to col. 8/line 29). 

Regarding claim 1 8, Sezan discloses "a method for generating video summary 
description data according to video summary description scheme by inputting original video, 
comprising: video analyzing step which is producing video analysis result by inputting the 
original video and then analyzing the original video; summary rule defining step which is 
defining the summary rule for selecting summary video interval; summary video interval 
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selecting step which is constituting summary video interval information by selecting the video 
interval capable of summarizing video contents from the original video by inputting said original 
video analysis result and said summary rule; and video summary describing step which is 
producing video summary description data according to the HierarchicalSummary DS by 
inputting the summary video interval information output by said summary video interval 
selecting step" (see claims 1-4 and 10, with a video analyzing step using analysis manager 70, 
summary rule using highlight summarizer 78 and keyframe summarizer 76 for selecting 
summary video interval or segments as shown in Fig. 2, and svunmary is displaying to the user as 
show in Figs. 7-12 by selecting frame or summary etc. on the screen). 

As for claim 1 9, in fiirther view of claim 1 8, Sezan discloses "wherein said 
HierarchicalSummary DS comprises at least one HighlightLevel DS which is describing 
highlight level, wherein said HighlightLevel DS comprises at least HighlightSegment DS which 
is describing highlight segment information constituting the summary video of the highhght 
level, wherein said HighlightSegment DS comprises VideoSegmentLocator DS describing time 
information or video itself of said corresponding highlight segment" (see claims 1-3 above). 

As for claim 20, in fiirther view of claim 18, Sezan fixrther discloses "wherein said video 
analyzing step comprises: feature extracting step which is outputting the types of features and 
video time interval at which those features are detected by inputting the original video and 
extracting those features; event detecting step which is detecting key events included in the 
original video by inputting said types of features and video time interval at which those features 
are detected; and episode detecting step which is detecting episode by dividing the original video 
into story flow base on the basis of said detected event", i.e., module 52 extracts information 
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based on types of features (col. 9/lines 9-26); and the event detecting step according to features, 
key events, program segment or event segment is discussed in claims 11-12 above. 

As for claims 21-23, in view of claim 18, these claims for the steps of "wherein said 
summary rule defining step provides the types of summary events, which are bases in selecting 
the summary video interval, after defining them to said video summary describing step"; 
"representative fi-ame extracting step which is providing the representative frame to said video 
summary describing step by inputting said summary video interval information and extracting 
representative frame"; and "representative sound extracting step which is providing the 
representative sound to said video summary describing step by inputting said summary video 
interval information and extracting representative sound" as described in earlier claims 4-8, but 
not limited to those claims but to the entire related features through out the claims. 

Regarding claims 24-31, these claims for "a computer-readable recording medium where 
a program is stored therein, the program is to execute: feature exfracting step which is outputting 
the types of features and video time interval at which those features are detected; event detecting 
step which is detecting key events included in the original video by inputting said types of 
features and said video time interval at which those features are detected; episode detecting step 
which is detecting episode by dividing the original video into story flow base on the basis of said 
detected key events; summary rule defining step which is defining the summary rule for selecting 
the summary video interval; summary video interval selecting step which is constituting 
summary video interval information by selecting the video interval capable of summarizing the 
video contents of the original video by inputting said detected episode and said summary rule; 
and video summary describing step which is generating video summary description data with 
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HierarchicalSummary DS by inputting the summary video interval information output by said 
summary video interval selecting step"; "a system for generating video summary description data 
according to video summary description scheme by inputting original video, comprising: video 
analyzing means for outputting video analysis result by inputting original video and analyzing 
the original video; summary rule defining means for defining the summary rule for selecting the 
summary video interval; summary video interval selecting means for constituting summary video 
interval information by selecting the video interval capable of summarizing the video contents of 
the original video by inputting said video analysis result and said summary rule; and video 
summary describing means for generating video summary description data with 
HierarchicalSummary DS by inputting the summary video interval information output by said 
summary video interval selecting means"; and "a computer-readable recording medium where a 
program is stored therein, the program is for fimctioning: featiire extracting means for outputting 
the types of features and video time interval at which those featiires are detected; event detecting 
means for detecting key events included in the original video by inputting said types of features 
and said video time interval at which those featiires are detected; episode detecting means for 
detecting episode by dividing the original video into story flow base on the basis of said detected 
key events; summary rule defining means for defining the summary rule for selecting the 
summary video interval; summary video interval selecting means for constitiiting summary video 
interval information by selecting the video interval capable of summarizing the video contents of 
the original video by inputting said detected episode and said summary rule; and video summary 
describing means for generating video summary description data with HierarchicalSummary DS 
by inputting the summary video interval information output by said summary video interval 
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selecting step" with same limitations are rejected for the reasons given in the scope of claims 1- 
17 as discussed in details above. 

Regarding claim 32, Sezan discloses "a video browsing system in a server/client 
circumstance, comprising:, a server which is equipped with video summary description data 
generation system which generates video summary description data on the basis of 
HierarchicalSummary DS by inputting original video and links said original video and video 
summary description data; and a client which is browsing and navigating video by overview of 
said original video and access to the original video of said server using said video simimary 
description data" (Fig. 2 for a server/client system, and col. 3/line 15 to col. 5/line 32 for 
methods of navigation or browsing video by overview of the original video and the user can 
access to the server using video summary description data, as shown in Figs. 4-12, and claims 1- 
10 above for already addressed limitations). 

Conclusion 

6. The prior art inade of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Madrane (U.S. Pat. No.6,573,907 Bl) discloses a network distribution and management of 
interactive video and multi-media contaimers. 

Appeh et al. (U.S. Pat. No.6,601,026 B2) disclose an information retrieval by natural language 
querry. 

Goldberg et al. (U.S. Pat. No.5,963,203) disclose an interactive video icon with designated 
viewing position. 
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7. Any response to this action should be mailed to: 



Commissioner of Patents and Trademarks 
Washington, D.C. 20231 



or faxed to: 



(703) 872-9306, (for Technology Center 2600 only) 

8. Any inquiry concerning this commimication or earUer communications from the examiner 
should be directed to Krista Kieu-Oanh Bui whose telephone number is (703) 305-0095. The 
examiner can normally be reached on Monday-Friday from 9:00 AM to 6:30 PM, with alternate 
Fridays off. 

If attempts to reach the examiner by telephone are unsuccessfiil, the examiner s 
supervisor, Andrew Faile, can be reached on (703) 305-4380. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to Technology Center 2600 Customer Service Office whose telephone number 
is (703) 306-0377. 




Krista Bui 
Art Unit 2611 
April 28, 2004 
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